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We consider maximum likelihood estimation for both causal and 
noncausal autoregressive time series processes with non-Gaussian en- 
stable noise. A nondegenerate limiting distribution is given for max- 
imum likelihood estimators of the parameters of the autoregressive 
model equation and the parameters of the stable noise distribution. 
The estimators for the autoregressive parameters are n 1//Q! -consistent 
and converge in distribution to the maximizer of a random function. 
The form of this limiting distribution is intractable, but the shape 
of the distribution for these estimators can be examined using the 
bootstrap procedure. The bootstrap is asymptotically valid under 
general conditions. The estimators for the parameters of the stable 
noise distribution have the traditional n 1//2 rate of convergence and 
are asymptotically normal. The behavior of the estimators for finite 
samples is studied via simulation, and we use maximum likelihood 
estimation to fit a noncausal autoregressive model to the natural log- 
arithms of volumes of Wal-Mart stock traded daily on the New York 
Stock Exchange. 

1. Introduction. Many observed time series processes appear "spiky" 
due to the occasional appearance of observations particularly large in abso- 
lute value. Non-Gaussian a-stable distributions, which have regularly vary- 
ing or "heavy" tail probabilities (P(|X| > x) ~ (constant) x~ a , x > 0, < a < 
2), are often used to model these series. Processes exhibiting non-Gaussian 
stable behavior have appeared, for example, in economics and finance (Em- 
brechts, Kluppelberg and Mikosch [18], McCulloch [25] and Mittnik and 
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Rachev [28]), signal processing (Nikias and Shao [29]) and teletraffic engi- 
neering (Resnick [32]). 

The focus of this paper is maximum likelihood (ML) estimation for the 
parameters of autoregressive (AR) time series processes with non-Gaussian 
stable noise. Specific applications for heavy-tailed AR models include fitting 
network interarrival times (Resnick [32]), sea surface temperatures (Gal- 
lagher [20]) and stock market log-returns (Ling [24]). Causality (all roots 
of the AR polynomial are outside the unit circle in the complex plane) is 
a common assumption in the time series literature since causal and non- 
causal models are indistinguishable in the case of Gaussian noise. However, 
noncausal AR models are identifiable in the case of non-Gaussian noise, 
and these models are frequently used in deconvolution problems (Blass and 
Halsey [3], Chien, Yang and Chi [10], Donoho [16] and Scargle [36]) and 
have also appeared for modeling stock market trading volume data (Breidt, 
Davis and Trindade [5]). We, therefore, consider parameter estimation for 
both causal and noncausal AR models. We assume the parameters of the 
AR model equation and the parameters of the stable noise distribution are 
unknown, and we maximize the likelihood function with respect to all pa- 
rameters. Since most stable density functions do not have a closed- form 
expression, the likelihood function is evaluated by inversion of the stable 
characteristic function. We show that ML estimators of the AR parameters 
are n 1//o -consistent (n represents sample size) and converge in distribution 
to the maximizer of a random function. The form of this limiting distribu- 
tion is intractable, but the shape of the distribution for these estimators can 
be examined using the bootstrap procedure. We show the bootstrap pro- 
cedure is asymptotically valid provided the bootstrap sample size m n — ► oo 
with m n /n — ► as n — > oo. ML estimators of the parameters of the stable 
noise distribution are n 1//2 -consistent, asymptotically independent of the AR 
estimators and have a multivariate normal limiting distribution. 

Parameter estimation for causal, heavy-tailed AR processes has already 
been considered in the literature (Davis and Resnick [14], least squares esti- 
mators; Davis [11] and Davis, Knight and Liu [12], least absolute deviations 
and other M-estimators; Mikosch, Gadrich, Kliippelberg and Adler [27], 
Whittle estimators; Ling [24], weighted least absolute deviations estima- 
tors). The weighted least absolute deviations estimators for causal AR pa- 
rameters are re 1//2 -consistent, and the least squares and Whittle estimators 
are (n/ Inn) -^"-consistent, while the unweighted least absolute deviations 
estimators have the same faster rate of convergence as ML estimators, n l l a . 
Least absolute deviations and ML estimators have different limiting distri- 
butions, however, and simulation results in Calder and Davis [8] show that 
ML estimates (obtained using the stable likelihood) tend to be more effi- 
cient than least absolute deviations estimates, even when the AR process 
has regularly varying tail probabilities but is not stable. Theory has not yet 



ML ESTIMATION FOR q-STABLE AR PROCESSES 



3 



been developed for the distribution of AR parameter estimators when the 
process is noncausal and heavy-tailed. 

In Section 2, we discuss properties of AR processes with non-Gaussian 
stable noise and give an approximate log-likelihood for the model param- 
eters. In Section 3, we give a nondegenerate limiting distribution for ML 
estimators, show that the bootstrap procedure can be used to approximate 
the distribution for AR parameter estimators, and discuss confidence interval 
calculation for the model parameters. Proofs of the Lemmas used to establish 
the results of Section 3 can be found in the Appendix. We study the behav- 
ior of the estimators for finite samples via simulation in Section 4.1 and, in 
Section 4.2, use ML estimation to fit a noncausal AR model to the natu- 
ral logarithms of volumes of Wal-Mart stock traded daily on the New York 
Stock Exchange. A causal AR model is inadequate for these log-volumes 
since causal AR residuals appear dependent. The noncausal residuals ap- 
pear i.i.d. (independent and identically distributed) stable, and so the fitted 
noncausal AR model appears much more suitable for the series. 

2. Preliminaries. Let {Xt} be the AR process which satisfies the differ- 
ence equations 

(2.1) MB)X t = Z t , 

where the AR polynomial (fro(z) := I — 4>m z — • ■ • — (po p z p ^ for \z\ = 1, B 
is the backshift operator (B k Xt = Xt~k, k = 0, ±1, ±2, . . .), and {Zt} is an 
i.i.d. sequence of random variables. Because (j>o(z) ^ for \z\ = 1, the Laurent 
series expansion of 1/4>q(z), 1/(J)q(z) = X^-oo ^Pj 2 ^ exists on some annulus 
{z : a" 1 < \z\ < a}, a > 1, and the unique, strictly stationary solution to (2.1) 
is given by X t = Y^jL-oo^j^t-j (see Brockwell and Davis [6], Chapter 3). 
Note that if (fio(z) ^ for \z\ < 1, then tpj = for j < 0, and so {X t } is said 
to be causal since X t = J2 < jLo' 1 PjZt-j, a function of only the past and present 
{Zt}. On the other hand, if 4>o(z) / for \z\ > 1, then Xt = YlJLoili-jZt+j-, 
and {X t } is said to be a purely noncausal process. In the purely noncausal 

case, the coefficients {i/ij} satisfy (1 — fioiz 4>OpZ p )(ipo + tp-tz _1 H ) = 

1, which, if <fiop / 0, implies that — ^-l = • • • = tpi- p = and if)- p = —(t>Q p l - 
To express <fio(z) as the product of causal and purely noncausal polynomials, 
suppose 

(2.2) Mz) = (1 " e 01 Z 0Or o * r °)(l - Oo,r +lZ ^ro+so^ ), 

where ro + «o = P, #o( z ) := 1 — Qoi? — ■ ■ ■ — 9o ro z r ° ^ for \z\ < 1, and 

9q(z) := 1 — 9q^ q+ iz 9q^ 0+Sq z s ° / for \z\ > 1. Hence, 9\{z) is a causal 

polynomial and 9q(z) is a purely noncausal polynomial. So that (f>o( z ) has a 
unique representation as the product of causal and purely noncausal polyno- 
mials 9q(z) and 9q(z), if the true order of the polynomial <f>o{z) is less than 
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p (if (pop = 0), we further suppose that #o,r +so when so > 0. Therefore, 
if the true order of the AR polynomial 4>o(z) is less than p = ro + sq, then 
the true order of 0q(z) is less than ro, but the order of 0q(z) is so- 

We assume throughout that the i.i.d. noise {Zt} have a univariate sta- 
ble distribution with exponent ao S (0,2), parameter of symmetry |/?o| < 1, 
scale parameter < Co < 00, and location parameter fiQ £ R. Let to = 
(ao, A))°o> A*o)' • By definition, nondegenerate, i.i.d. random variables {St} 
have a stable distribution if there exist positive constants {a n } and con- 

stants {b n } such that a n (S\ + • • • + S n ) + b n = S\ for all n. In general, stable 
distributions are indexed by an exponent a 6 (0,2], a parameter of symme- 
try \fi\ < 1 5 a scale parameter < a < 00 and a location parameter /iGK. 
Hence, to is in the interior of the stable parameter space. If /3 = 0, the stable 
distribution is symmetric about fi, and, if a = 1 and (3 = 0, the symmetric 
distribution is Cauchy. When a = 2, the stable distribution is Gaussian with 
mean \i and standard deviation y/2a. Other properties of stable distributions 
can be found in Feller [19], Gnedenko and Kolmogorov [21], Samorodnitsky 
and Taqqu [35] and Zolotarev [38]. 

Since the stable noise distribution has exponent ao < 2, 



(2.3) 



lim x ao V(\Z t \ >x)= c(a )cJn , 

x — >oo 

with c(a) := 



t~ a sin(t) dt 



-1 



(Samorodnitsky and Taqqu [35], Property 1.2.15). Following Properties 1.2.1 
and 1.2.3 in Samorodnitsky and Taqqu [35], X t = J2'j^=-oo' t l J jZt-j also has 
a stable distribution with exponent ao and, hence, the tail probabilities for 
the AR process {Xt} are also regularly varying with exponent ao- It follows 
that E\X t \ 5 < 00 for all 5 G [0, a ) and E\X t \ s = 00 for all 5 > a . 
The characteristic function for Zt is 



(p (s) := E{exp(isZ t )} 



(2.4) 



cxp 



-cr, 



\ s \ 



i) 



if «o / 1, 



exp^ — (Jq\s\ 
if ao 



1 + i/3 — (signs)ln(cr |s| 

7T 



and so the density function for the noise can be expressed as /(z;to) = 
(27T)" 1 /f^ exp(— izs)ipo(s) ds. No general, closed-form expression is known 
for /, however; although, computational formulas exist that can be used 
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to evaluate / (see, e.g., McCulloch [26] and Nolan [30]). It can be shown 
that f(z;T ) = aQ 1 f(aQ 1 (z - fi );(a ,(3 , 1,0)'), /(•; (a ,/3 , 1,0)') is uni- 
modal on R (Yamazato [37]), and f(z; (a, (3, 1, 0)') is infinitely differen- 
tiable with respect to (z,a,(3) on R x (0,2) x (—1,1). There are alterna- 
tive parameterizations for the stable characteristic function <po (see, e.g., 
Zolotarev [38]), but we are using (2.4) so that the noise density function 
is differentiable with respect to not only z on 1 but also (a, (3, a, fi)' on 
(0,2) x (—1,1) x (0,oo) x (— oo,oo). From asymptotic expansions in Du- 
Mouchel [17], if &s := {r = (a, P, a, fi)' : ||r - r || < 5}, then for 5 > suffi- 
ciently small we have the following bounds for the partial and mixed partial 
derivatives of ln/(z;r) as \z\ — ► oo: 



(2.5) 



(2.6) 



(2.7) 



(2.8 



(2.9) 



(2.10) 



sup 

n<5 



sup 



d 2 In f(z;r) 



dz 2 

= 0(\z\~ 2 ) 

dhxf(z]T) 



+ sup 
(is 



d 2 ln/( 



z;t\ 



sup 
(is 



dz dfi 
91n/(z;r) 



sup 
Us 



# In f(z;r) 



dfj? 



+ sup 

Us 



+ sup 
Sis 



d 2 In f(z;r) 



dz da 
d 2 In f(z;r) 



dfi 



+ sup 
(is 



+ sup 
(is 



d 2 In f{z;r) 



dz dp 



d 2 In f(z;r) 



dfidix 



da dfi 



0(\z\ 



sup 
(is 



sup 
(is 



d 2 \nf(z;r) 



dz da 
dlnf(z;r) 



+ sup 
(is 



d 2 lnf(z;r) 



df3 



sup 
(is 



da d[i 
d\nf(z;r) 



+ sup 
(is 



d 2 lnf(z;r) 



dp da 



da 



+ sup 
(is 



+ sup 
(is 



CKM-'lnM), 
d 2 lnf(z;r) 



d 2 ln/( 



z:t 



sup 
(is 



sup 
(is 



dlnf(z;r) 



da 

= 0(ln\z\), 
d 2 \nf(z;r) 



sup 
(is 



d 2 In f(z;r) 



da dp 



da 2 



+ sup 

(is 



dp 2 

= 0(1), 
d 2 In f(z;r) 



da da 



da 2 



0([ln\z\ 



From (2.1) and (2.2), Z t = (1 - 6» i# 



^0r o -B r °)(l — #0,r o +l-B 



'0,ro+so 
?1* 



B s °)Xt. Therefore, for arbitrary autoregressive polynomials 9\z 



1 



„z r and 0*(z) = 1 



'r+s 



z s , with r + s = p, 
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0t(z) / for \z\ < 1, 6*{z) / for \z\ > 1, and 6» r+s / when s > 0, we 
define 

(2.11) z t (e, s ) = (i-e 1 B e r B r )(i - e r+1 B e r+s B s )x t , 

where 6 := (6>i, . . . , P )' . Let 6q = (6qi, . . . , 9q p )' denote the true parameter 
vector and note that {Z t (6o, so)} = {Zt}. Now, let r] = (771 rjp+i)' = 
(6i,...,6 p ,a,P,(T,ii,y = (0',t')', and let T7 = (r/oi, . . . ,r?o,p+4)' = (^o> r d)'- 
From Breidt et al. [4], given a realization {A" t }™ =1 from (2.1), the log- 
likelihood of rj can be approximated by the conditional log-likelihood 

n 

(2.12) C(rj,s)= J2 [lnf(Z t (e,s);r)+ln\e p \I{s>0}}, 

t= P +i 

where {Z t (0, s)}™ =p+1 is computed using (2.11) and /{•} represents the in- 
dicator function (see [4] for the derivation of C). Given {X^}™ =1 and fixed 
p, we can estimate so, the order of noncausality for the AR model (2.1), 
and rj by maximizing C with respect to both s and rj. If the function g is 
defined so that 



g(e,s) = [g J (9,s)]% 1 

(2.13) 



9j(9, s ) 



3 



X! Qj-k^p-s+k, .7 = 1; ■•■iP — s i 



k=l 
3 



Qj-kQp-s+k, j =p- s + i,...,p, 

k=j—p+s 



with 9q = — 1 and ^fc = whenever fc ^ {0, . . . then an estimate of O := 
(0oi j • • • 1 ^Op)' can be obtained using the MLEs of so and 6q and the fact that 
00 = 9(00, s o)- A similar ML approach is considered in [4] for lighter-tailed 
AR processes. 



3. Asymptotic results. In this section, we obtain limiting results for 
maximizers of the log-likelihood C. But first, we need to introduce some 
notation and define a random function P^(-). The ML estimators of Oq con- 
verge in distribution to the maximizer of W(-). 

Suppose the Laurent series expansions for 1/0,!, (z) = 1/(1 — 9q\z — ■ ■ ■ — 

e 0ro z ro ) and l/05(z) = l/(l-0 Oj r o +i2 0o,r o +s o z s °) are given by l/e\{z) = 

££o*j* j and l/0* Q {z)=Y.T= So XjZ- j - From (2.11), 

dZ t (0,s) _f-9*(B)X t - j , J = l,...,r, 

{ -> &e j ~\-e\B)x t+r _ j: j = r + i,..., P , 
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and so, for u = (u±, . . . , u p )' G MP, 
w dZ t (Oo,s ) = _ Ui0 * {B)Xt i Ur X(B)X t . ro - u ro+1 9l(B) 



t-i 



Up el(B)x t ^ S0 

-«i(l/eJ(B))Z t _i u ro (l/6t(B))Z t _ ro 

- Uro+1 (i/e* (B))z t ^ Up (i/e* (B))z t - S0 

oo oo 

-Ml J2 KjZt-l-j U ro J2 KjZt-ro-j 

j=0 j=0 

oo oo 

- U ro+ i XjZt-l+j U p ^ XjZt-s +j- 

j = S j = SQ 



Therefore, if 



(3.2) 2^ Cj(u)Zt-3 :=u 



80 

j = -oo 

then c (u) = -u p x 30 I{s > 0} = u p 9^I{s > 0}, ci(u) = -Ui7r I{r > 0} = 
-«il{r > 0}, c_i(u) = -u p X2l{ s o = 1} - (%-iXso + u p Xs Q +i)I{ s o > 1}, 
and so on. Since {itj}°fL and {xj}j*L S0 decay at geometric rates (Brockwell 
and Davis [6], Chapter 3), for any u G MP, there exist constants C(u) > 
and < D (u) < 1 such that 

(3.3) | Cj (u)| < C(u)[£>(u)]l'l Vj G {. . . , -1, 0, 1, . . .}. 

We now define the function 

oo 

W ( u ) = E EO n /(^.i + [c(ao)] 1/Q Vo Cj (u)5 fc r- 1/Qo ;ro) 

(3.4) 

- ln/(Z fc)J -;T )}, 

where 

• {Zkj}k,j is an i.i.d. sequence with Z^j = Z\, 

• c(-) was defined in (2.3), 

• {5 k } is i.i.d. with P(5 k = 1) = (1 + /?o)/2 and P(5 k = -1) = 1 - (1 + ft,)/2, 

• T k = E\ + ■ ■ ■ + E k , where {E k } is an i.i.d. series of exponential random 
variables with mean one, and 

• {Zfcj}, {<5fc} and {E k } are mutually independent. 

Note that (1 + A))/2 = lim a; ^ 00 [P(Zi > x)/P(|Zi| > a;)] (Samorodnitsky and 
Taqqu [35], Property 1.2.15). Some properties of W(-) are given in the fol- 
lowing theorem. 
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Theorem 3.1. With probability one, the function W (u) defined in (3.4) 
is finite for all u£K p and has a unique maximum. 

PROOF. Let ueK p and observe that 

xxrf \ X^Y^r^ Ml/ao I \x t-~ V"o ^hi /(Z fcj (u); T ) 

oo 

=EEwf / %oc 3 (u)4 

fc=li^0 

'91n/(Z fc| j(u);r ) d\nf(Z kJ ; r ) 



x r,T 1/ao 



<9z <9z 



+ EE[^o)] 1/a v„ Cj (u)4(r,: 1/a « - fc -i/ V ln/( 5 fcj;ro) 
+ EE[%°)] 1/tt °%-(")^- 1/a ° aiD /( f * Ji To) , 

fc=ii#o 0Z 

where j(u) lies between Zj.j and Z^j + [^(ao)] 1 ^ ^^^ 11 )^^^ Since 
< [c(ao)] ly ' ao <To < °°7 by Lemmas A.1-A.3 in the Appendix, |W(u)| < oo 
almost surely. It can be shown similarly that sup^n^ |W(u)| < oo almost 
surely for any T G (0,oo) and, therefore, P(n7?=i{sup|| u || <T | W(u) | < oo}) = 
1. 

Since /(-; r o) is unimodal and differentiable on R, with positive proba- 
bility, ln/(Zi + -;tq) is strictly concave in a neighborhood of zero, and so, 
by Remark 2 in Davis, Knight and Liu [12], W(-) has a unique maximum 
almost surely. □ 

We now give nondegenerate limiting distributions for ML estimators of 
rj = (0q, t' Y = (0oi, • • • , Go P , do, Po, Co, no)' and estimators of the AR param- 
eters 4> = {(poi, ■ ■ -,(po p y in (2.1). 

Theorem 3.2. There exists a sequence of maximizers i) ML = (^mli^ml)' 
of C(-,sq) in (2.12) such that, as n^oo, 

(3.5) nV^ML-flo)^ and n 1 ' 2 ^ ML - t ) ^ Y ~ N(0, JT^tq)), 

where £ is the unique maximizer of W(-), £ and Y are independent, and 
I ( T ) : =-[ E {d 2 \nf(Z 1 ;T)/(dT i dT j )}] i j e { lt _ >i }. In addition, ifcb ML := g(0 M L,s ), 
with g as defined in (2.13), then 

(3.6) n 1/ao (0ML-0o)^E(0o)*, 
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where 



(3.7) 



£(0) :-- 



dgi(9,s ) 

80! 



dg p (6,s ) 

80! 



dgi(e,s ) 



dg p (0,s ) 



and gi,...,g p were also defined in (2.13). 



Since to is in the interior of the stable parameter space, given i.i.d. ob- 
servations {Zt}f = i, ML estimators of To are asymptotically Gaussian with 
mean to and covariance matrix I~ 1 (To)/n (see DuMouchel [17]). The esti- 
mators tml, therefore, have the same limiting distribution as ML estimators 
in the case of observed i.i.d. noise. Nolan [31] lists values of I _1 (-) for differ- 
ent parameter values. 

For uGff and v G M 4 , let W n (u, v) = C(r] Q + (n^/^u', n^V)', s ) - 
C(t] q ,sq), and note that maximizing C(r], so) with respect to rj is equivalent 
to maximizing W n (u, v) with respect to u and v if u = n 1 / Q °(0 — 6q) and 
v = ?i 1 / 2 (t — to). We give a functional convergence result for W n in the 
following theorem, and then use it to prove Theorem 3.2. 

Theorem 3.3. isruoo, W n {u, v) W(u) + v'N - 2~VI(t )v on 
C(MP +4 ), where N~ JV(0,I(t )) is independent ofW(-), andC{W+ 4 ) rep- 
resents the space of continuous functions on M p+4 , where convergence is 
equivalent to uniform convergence on every compact subset. 

Proof. For u£l p and v G M 4 , let 

W*(u,v)= jr \\nf[z t + n- 1 ' ao Y. c M) Z t~vrA - ln/(Z t ;T )} 
t=p+l { \ j^o / ) 

_V_ » din f(Z t ;r ) 

Since 

W„(u,v)-W^(u,v) 



dr 



t=p+l \ 



n 



-l/ao 



j^o J 



10 



B. ANDREWS, M. CALDER AND R. A. DAVIS 

#o P + n~ 1/Q °% 



v' " ain/(Z f ;r ) 



/{so > 0}, 



W n (u,v) - W*(u,v) + 2- 1 v'I(r )v = 0p (l) on C(W +i ) by Lemmas A.4- 

A.7. So, the proof is complete if W*(u, v) -4 pF(u) + v'N on C(M P+4 ). 
For uGlRP, let 



(3.8) W*(u) = £ 

t=p+l L 

and, for v 6 R 4 , let 



In / Z t + n" 1 ^ ^ Cj (u)Z t ^ ; t - In /(Z t ; r c 



„, , s v' ^ 31nf(Z+;rn) 
(3.9) W = ^ E g T • 

By Lemma A.8, for fixed u and v, (W^(u),T n (v))' £ (f(u),v'N)' on 
R 2 , with W(u) and v'N independent. Consequently, W*(u, v) = W^(u) + 

T n (v) A W(u) + v'N on R. Similarly, it can be shown that the finite di- 
mensional distributions of W^(u,v) converge to those of W(u) + v'N, with 
W(-) and N independent. For any compact set K\ CK P , is tight 

on C(K\) by Lemma A. 12 and, for any compact set Ki C M 4 , {T n (-)} is 
tight on CiK^) since T n (y) is linear in v. Therefore, by Theorem 7.1 in 

Billingsley [2], W*(u,v) = W&(u) + T„(v) 4 W(u) + v'N on C(M P+4 ). □ 



Proof of Theorem 3.2. Since W„(u,v) W(u)+ v'N- 2- 1 v'I(t )v 
on C(IR P+4 ), £ uniquely maximizes W(-) almost surely, and Y = I~ 1 (to)N 
uniquely maximizes v'N — 2~ 1 v'I(to)v, from Remark 1 in Davis, Knight 
and Liu [12], there exists a sequence of maximizers of W n (-,-) which con- 
verges in distribution to (£', Y')'. The result (3.5) follows because C(r/, sq) — 
C(rj ,s ) = W n {n l t c,0 (0 - Qa),n l / 2 {T -r )). By Theorem 3.3, £ and Y are 
independent. 

Using the mean-value theorem, 

n l/ao (<£ml " 0o) = nlh0 GK^ML, s ) - g(e ,8 )) 



(3.10) 



dgi(0*i,s ) 

80! 

dg P {0* pi SQ) 
dOi 

X7i 1/a °(0ML 



d6 n 



dg P {0* p ,s a ) 
d9 n 
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p 

where Q\, . . . ,0* lie between Oml and 6q. Since 9ml —> #o and S(-) is con- 
tinuous at 9q, (3.10) equals Yl(6o)n l l ao {6ml — #o) + o p (l). Therefore, the 
result (3.6) follows from (3.5). □ 

Since the forms of the limiting distributions for 0ml and 4>ml in (3-5) 
and (3.6) are intractable, we recommend using the bootstrap procedure to 
examine the distributions for these estimators. Davis and Wu [15] give a 
bootstrap procedure for examining the distribution of M-estimates for the 
parameters of causal, heavy-tailed AR processes; we consider a similar pro- 
cedure here. Given observations {X t }" =1 from (2.1), 6ml from (3.5), and cor- 
responding residuals {Zt(0ML, s o)}t=p+i obtained via (2.11), the procedure 
is implemented by first generating an i.i.d. sequence {Z£ }^ from the empir- 
ical distribution for {Zt(0MLo s o)}t=p+i- A bootstrap replicate X{ , . . . , X^ n 
is then obtained from the fitted AR(p) model 

(3.H) e Uh {B)e M L{B)x; = z;, 

where 9 ML (z) := 1 - 9 1M lz 9 r0)M LZ r ° and 9 M l( z ) '■= 1 _ 4- +i,ML£ - 

9r 0+S0 ,MLZ S0 (let Zt = for t$ {1, . . . ,m n }). Finally, with Z?(0,s) := 

(i - q x b e r B r ){i - e r+1 B e r+s B s )x$ for e = {Ox, . . . , o p y e w 

and r + s = p, a bootstrap replicate mn of 0ml can be found by maximizing 

m n 

C* mn (0,s ):= J2 [rn/(^(0,s o );f ML ) + ln|^|I{ So >O}] 

t=p+i 

with respect to 0. The limiting behavior of mn , along with that of </> mn := 

g(O mn ,so) (a bootstrap replicate of ^> ML ), is considered in Theorem 3.4. 
To give a precise statement of the results, we let M P (M?) represent the 
space of probability measures on MP and we use the metric d p from Davis 
and Wu ([15], page 1139) to metrize the topology of weak convergence 

on Ai p {MP). For random elements Q n and Q of M P (MP), Q n — ► Q if and 

P p 
only if dp(Q n , Q) — > on R, which is equivalent to / Rp /ij dQ n — ► / KP hj dQ 

on M for all j £ {1, 2, . . .}, where {hj}°^ =1 is a dense sequence of bounded, 
uniformly continuous functions on W. By Theorem 3.4, P{rrh/ aMh (0 mn — 
0ml) G -|Xi,...,X n ) converges in probability to P(£ G •) on A / i p (M p ) [£ 
represents the unique maximizer of W(-)], and a similar result holds for 

ml/^^-^ML)- 

Theorem 3.4. If, as n ^ oo, m n — > oo with m n /n — > 0, then there exists 
a sequence of maximizers 6 rrin o/£* (-,so) smc/j i/jai 

n^ /6M1 (t - ^ml) £'l4..,In)-^e •) 
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on M P (W) and, if 4>* mn = g(6* mn ,s ), then 



(3.12) P(ml/ & ^ (0^ - ML ) G -IX!,. ..,X n )$ P(E(.0 )Z G •) 
on 7W p (M p ) /£(•) was de/med in (&7j/. 

PROOF. Since Z*(0,s) = (l-^fl 9 r B r )(l-6 r+1 B r+s -B s ) x 

X t *, following (3.1), for u = (ui, ... , u p )' G W, 

u qq = -uiO Mh {B)X t _ l u ro e ML {B)X t _ ro 

- uro+idLwxu Up el h (B)x;_ so 

= -mii/e^mzu u ro (i/el L (B))z^ ro 

-uro+iO-fotsLmzu u p (i/9* ML (B))z;_ S0 . 

We define the sequence {cj(u)}j^ = _ OQ so that 

(3.13) ^ c J (u)Z t _ j = u — . 

j=-oo 

Also, for u G M p , 



(3.14) 



*=p+i 



In/ Z* t + m-V«o ^ Cj (u) Z£_j ; To - In f(Z t *; r c 



and 

(3.15) W mn (u) := C* mn (e ML + m~ 1 /°o U) Sq) _ £ ^ (0mLj *))• 

Now, let _M P (C(R P )) represent the space of probability measures on C(M P ), 
and let do metrize the topology of weak convergence on M P (C(M P )). That 

is, for random elements L n and L of .M P (C(R P )), L n — ► L if and only if 

do(L n ,L) -* on R, and there exists a dense sequence {^/I^Li of bounded, 

continuous functions on C(M. P ) such that do(L n ,L) — ► is equivalent to 

~ P ~ 

fens?) hj dL n — ► Scnstp) hj dL on R for all j G {1,2, . . .}. We now show that, 
if L n (.) := P(VF mn G . . . ,X n ) and 4(-) := P(Wl n e -\X U . . . ,X n ), then 

L n - L\ 4 on 7W p (C(]R p )). Following the proof of Theorem 2.1 in [15], it 
suffices to show that for any subsequence {n^} there exists a further sub- 
sequence {n^/} for which L ny — L^ n l ^+ relative to the metric do, which 



holds if, for almost all realizations of {X t }, W mn ; (-) — W^ n ; (-) on 
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C(R P ). By Lemma A. 13, for any subsequence, any Tg {1,2,...} and any 
K G {1,1/2,1/3,...}, there exists a further subsequence {n k , K } for which 
P(supi| u ||< r |# m Tk (u)-W^ Th (u)\ > k\Xi,...,X t,k)^0. Usingadiag- 

onal sequence argument, it follows that there exists a subsequence {n^/} 
of {n k } for which P(sup|| u ||< T \W mny (u) - W^, (u)| > k\X 1: . . 

for almost all {X t } and any T, k > and, thus, W mny (•) — / (•) on 
C(W) for almost all {X t }. 

Following the proof of Theorem 3.1 in [15], L^(-) = P{W^ ln G 

X n )^P(WG-) on M p (C(IRP)), and so L n (-)=P(W mn e-\X 1 ,...,X n )-^ 
P{W G •) on M P (C(W)) also. Therefore, because £^(0, s ) -£™„(0ml, s ) = 
W mn (ml/ ao x {6 — 6ml)) and £ uniquely maximizes W(-) almost surely, it 
can be shown that there exists a sequence of maximizers 6 mn of (-,so), 

such that P(my ao (0^ n - M l) 6 . . . , X n ) 4 P(£ G •) on M P {W) (the 
proof is similar to that of Theorem 2.2 in [15]). Since 

m l J & ^{6* mn - 6ml) - ni l J^{6* mn - ML ) 

( mi /an ln(m n )\ l/a /a* a \ 

V m„' {a* n y / 

where a* lies between oml and ao, and n^ 2 («ML-oo) = O p (l), P(|| (mi'" Ml — 

m l r! ao ){6* mn -6ML)\\ > k\Xi, . . . ,X n ) 4 Ofor any k > 0. Hence, P{m 1 n /&ML {6* rin - 
p 

6ml) G -\Xi, . . . ,X n ) —> P(£ G •) on A4 P (W). The mean-value theorem can 
be used to show that (3.12) holds. □ 

Thus, ml/ aML (6* mn - 6 M l) and ml /aML {4>* mn - 4> ML ) , conditioned on {X t } t " =1 , 

have the same limiting distributions as n l l a ° {6ml — 0o) and Ji 1 ^ (0ml ~~ 
O ), respectively. If n is large, these limiting distributions can, therefore, be 
approximated by simulating bootstrap values of 0™ and <\> mn , and looking 
at the distributions for mj/ aML {& mn — 6ml) and mj/ aML (0 mn — 0ml)- In 
principle, one could also examine the limiting distributions for n l l a ° {6ml — 
6q) and ^ 1//q °(0ml ~~ 0o) by simulating realizations of W(-), with the true 
parameter values 0o and to replaced by estimates, and by finding the cor- 
responding values of the maximizer £, but this procedure is much more 
laborious than the bootstrap. Confidence intervals for the elements of 6q 
and O can be obtained using the limiting results for 6ml and 0ml m (3.5) 
and (3.6), bootstrap estimates of quantiles for the limiting distributions and 
the estimate qml of ao- 
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For the elements of To, confidence intervals can be directly obtained from 
the limiting result for tml in (3.5). Because is continuous at To and 

tml —>■ To, I (f ml) is a consistent estimator for I (to) which can be used 
to compute standard errors for the estimates. 

4. Numerical results. 

4.1. Simulation study. In this section we describe a simulation exper- 
iment to study the behavior of the ML estimators for finite samples. We 
did these simulations in MATLAB, using John Nolan's STABLE library 
(http://academic2.american.edu/~jpnolan/stable/stable.html) to generate sta- 
ble noise and evaluate stable densities. The STABLE library uses the algo- 
rithm in Chambers, Mallows and Stuck [9] to generate stable noise and the 
algorithm in Nolan [30] to evaluate stable densities. 

For each of 300 replicates, we simulated an AR series of length n = 500 
with stable noise and then found r) ML = (^mlj^ml)' by maximizing the 
log-likelihood C in (2.12) with respect to both s € {0,...,p} and rj. To 
reduce the possibility of the optimizer getting trapped at local maxima, 
for each s 6 {0, ...,p}, we used 1200 randomly chosen starting values for 
rj. We evaluated the log-likelihood at each of the candidate values and, for 
each s 6 {0, ... ,p}, reduced the collection of initial values to the eight with 
the highest likelihoods. Optimized values were found using the Nelder-Mead 
algorithm (see, e.g., Lagarias et al. [23]) and the 8(p + 1) initial values as 
starting points. The optimized value for which the likelihood was highest 
was chosen to be ?)mli an d then ^) ML was computed using (2.13). In all 
cases, C was maximized at s = s$, so the true order of noncausality for the 
AR model was always correctly identified. 

We obtained simulation results for the causal AR(1) model with param- 
eter O = 0.5, the noncausal AR(1) model with parameter O = 2.0 and 
the AR(2) model with parameter O = (—1-2,1.6)'. The AR(2) polynomial 
1 + 1.2z — 1.6z 2 equals (1 — 0.8z)(l + 2z), and so it has one root inside and 
the other outside the unit circle. Results of the simulations appear in Ta- 
ble 1, where we give the empirical means and standard deviations for the 
parameter estimates. The asymptotic standard deviations were obtained us- 
ing Theorem 3.2 and values for I _1 (to) in Nolan [31]. (Values for I _1 (-) not 
given in Nolan [31] can be computed using the STABLE library.) Results 
for symmetric stable noise are given on the left-hand side of the table, and 
results for asymmetric stable noise with /3q = 0.5 are given on the right-hand 
side. In Table 1, we see that the MLEs are all approximately unbiased and 
that the asymptotic standard deviations fairly accurately reflect the true 
variability of the estimates oml, Pml, <5ml, and /1ml- Note that the values 
of ML , otML, Pml, and /1ml are less disperse when the noise distribution is 
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Table 1 

Empirical means and standard deviations for ML estimates of AR model parameters. 
The asymptotic standard deviations were computed using Theorem 3.2 and Nolan [31] 

Asymp. Empirical Asymp. Empirical 

std. dev. mean std. dev. std. dev. mean std. dev. 



4>oi = 


= 0.5 




0.500 


0.001 


ao = 


0.8 


0.051 


0.795 


0.040 


00 = 


0.0 


0.067 


0.000 


0.064 


CO = 


1.0 


0.077 


0.996 


0.068 


Mo = 


0.0 


0.054 


0.003 


0.057 


001 = 


= 0.5 




0.498 


0.019 


Q = 


1.5 


0.071 


1.499 


0.069 


00 = 


0.0 


0.137 


0.012 


0.142 


oo = 


1.0 


0.048 


0.997 


0.050 


Mo = 


0.0 


0.078 


-0.002 


0.074 


001 = 


= 2.0 




2.000 


0.004 




0.8 


0.051 


0.797 


0.041 


:> 


0.0 


0.067 


0.000 


0.066 


(TO = 


1.0 


0.077 


1.004 


0.072 


MO = 


0.0 


0.054 


0.004 


0.055 


001 = 


= 2.0 




2.003 


0.074 


QO = 


1.5 


0.071 


1.505 


0.074 


00 = 


0.0 


0.137 


0.008 


0.138 


0-Q = 


1.0 


0.048 


1.000 


0.056 


Mo = 


0.0 


0.078 


-0.006 


0.077 


001 = 


= -1.2 




-1.200 


0.004 


002 = 


= 1.6 




1.600 


0.004 


QO = 


0.8 


0.051 


0.798 


0.041 


00 = 


0.0 


0.067 


-0.001 


0.068 


CO = 


1.0 


0.077 


0.997 


0.073 


Mo = 


0.0 


0.054 


-0.002 


0.057 


001 = 


= -1.2 




-1.212 


0.083 


002 = 


= 1.6 




1.605 


0.065 


ao = 


1.5 


0.071 


1.502 


0.069 


00 = 


0.0 


0.137 


0.010 


0.128 


CO = 


1.0 


0.048 


0.999 


0.066 


Mo = 


0.0 


0.078 


-0.006 


0.078 



001 = 


= 0.5 




0.500 


0.001 


ao = 


0.8 


0.049 


0.799 


0.035 


00 = 


0.5 


0.058 


0.504 


0.060 


O"0 = 


1.0 


0.074 


0.995 


0.075 


Mo = 


0.0 


0.062 


-0.002 


0.066 


001 = 


= 0.5 




0.500 


0.018 


ao = 


1.5 


0.070 


1.500 


0.066 


0o = 


0.5 


0.121 


0.491 


0.121 


o~o = 


1.0 


0.047 


0.996 


0.047 


Mo = 


0.0 


0.078 


0.005 


0.082 


001 - 


= 2.0 




2.000 


0.004 


ao = 


0.8 


0.049 


0.795 


0.037 


00 = 


0.5 


0.058 


0.499 


0.060 


o-o = 


1.0 


0.074 


0.996 


0.072 


Mo = 


0.0 


0.062 


0.000 


0.063 


001 = 


= 2.0 




2.013 


0.073 


ao = 


1.5 


0.070 


1.497 


0.069 


0o = 


0.5 


0.121 


0.504 


0.119 


o-o = 


1.0 


0.047 


0.996 


0.061 


Mo = 


0.0 


0.078 


0.004 


0.079 


001 = 


= -1.2 




-1.200 


0.004 


002 = 


= 1.6 




1.600 


0.004 


ao = 


0.8 


0.049 


0.800 


0.039 


0o = 


0.5 


0.058 


0.502 


0.056 


o-o = 


1.0 


0.074 


0.997 


0.071 


Mo = 


0.0 


0.062 


-0.004 


0.064 


001 = 


= -1.2 




-1.204 


0.078 


002 = 


= 1.6 




1.598 


0.062 


ao = 


1.5 


0.070 


1.499 


0.071 


0o = 


0.5 


0.121 


0.509 


0.128 


o-o = 


1.0 


0.047 


0.997 


0.056 


Mo = 


0.0 


0.078 


0.000 


0.083 



heavier-tailed (ie., when ao = 0.8), while the values of <tml are more disperse 
when the noise distribution has heavier tails. Note also that the finite sam- 
ple results for tml do not appear particularly affected by the value of cf> , 
which is not surprising since ML and tml are asymptotically independent. 

Normal qq-plots show that, in all cases, &ml, Pml, <5"ml and //ml have ap- 
proximately Gaussian distributions. To examine the distribution for n 1//Q ° (0ml ~~ 
<M ), in Figure 1, we give kernel estimates for the density of n 1 ^ Q(1 (^i i ML — 4»oi) 
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when (<poi,ao,po,<r ,no) is (0.5,0.8,0,1,0), (0.5,0.8,0.5,1,0), (0.5,1.5,0,1,0) 
and (0.5, 1.5, 0.5, 1, 0). For comparison, we also included normal density func- 
tions in Figure 1; the means and variances for the normal densities are the 
corresponding means and variances for the values of n 1/ao {4>i ML -M - The 
distribution of n l / a ° (i^ml — </>oi) appears more peaked and heavier-tailed 
than Gaussian, but closer to Gaussian as olq approaches two. Similar behav- 
ior is exhibited by other estimators ^ml- 

4.2. Autoregressive modeling. Figure 2 shows the natural logarithms of 
the volumes of Wal-Mart stock traded daily on the New York Stock Ex- 
change from December 1, 2003 to December 31, 2004. Sample autocorre- 
lation and partial autocorrelation functions for the series are given in Fig- 
ure 3. Note that, even if a process has infinite second-order moments, the 
sample correlations and partial correlations can still be useful for identifying 
a suitable model for the data (see, e.g., Adler, Feldman and Gallagher [1]). 
Because the sample partial autocorrelation function is approximately zero 
after lag two and the data appear "spiky," it is reasonable to try model- 
ing this series {Xj}^ as an AR(2) process with non-Gaussian stable noise. 
Additionally, Akaike's information criterion (AIC) is smallest at lag two. 



<a> (b) 




-10 -5 5 10 15 20 25 -25 -20 -15 -10 -5 5 10 



(c) (d) 




-6-4 -2 2 4 -4 -2 2 4 

Fig. 1. Kernel estimates of the density for n 1 ^ ((/>i,ml — </>oi) when (4>oi, Go, flo, o~o, Mo) 
is (a) (0.5,0.8,0,1,0), (b) (0.5,0.8,0.5,1,0), (c) (0.5,1.5,0,1,0) and (d) (0.5,1.5,0.5,1,0), 
and normal density functions with the same means and variances as the corresponding 
values for n 1//Q ° (0i,ml — 4>oi)- 
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This supports the suitability of an AR(2) model for {Xt}. Note that AIC is 
a consistent order selection criterion for heavy-tailed, infinite variance AR 
processes (Knight [22]), even though it is not in the finite variance case. 

We fit an AR(2) model to {X t } by maximizing C in (2.12) with re- 
spect to both r] and s. The ML estimates are fj MIj = (6\, 02, a, (3, a, ft)' = 
(0.7380,-2.8146,1.8335,0.5650,0.4559,16.0030)', with s = l. Hence, the fit- 



o 




i 

so 



100 



- 1 — 

150 



200 



250 



t 



Fig. 2. The natural logarithms of the volumes of Wal-Mart stock traded daily on the New 
York Stock Exchange from December 1, 2003 to December 31, 2004- 



(a) ACF of X, 




(b) PACF of X, 




Fig. 3. (a) The sample autocorrelation function for {Xt} and (b) the sample partial 
autocorrelation function for {Xt}. 
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ted AR(2) polynomial has one root inside and one root outside the unit 
circle. The residuals from the fitted noncausal AR(2) model 

(4.1) (1 - 0.7380£)(1 + 2.8U6B)X t = (1 + 2.07665 - 2m72B 2 )X t = Z t 

and sample autocorrelation functions for the absolute values and squares 
of the mean-corrected residuals are shown in Figure 4(a)— (c). The bounds 
in (b) and (c) are approximate 95% confidence bounds which we obtained 
by simulating 100,000 independent sample correlations for the absolute val- 
ues and squares of 272 mean-corrected i.i.d. stable random variables with 
r = (1.8335,0.5650,0.4559,16.0030)'. Based on these graphs, the residu- 
als appear approximately i.i.d., and so we conclude that (4.1) is a satis- 
factory fitted model for the series {At}. A qq-plot, with empirical quan- 
tiles for the residuals plotted against theoretical quantiles of the stable 
r = (1.8335,0.5650,0.4559,16.0030)' distribution, is given in Figure 4(d). 
Because the qq-plot is remarkably linear, it appears reasonable to model the 
i.i.d. noise {Z t } in (4.1) as stable with parameter r = (1.8335, 0.5650, 0.4559, 
16.0030)'. Following the discussion at the end of Section 3, approximate 
95% bootstrap confidence intervals for 0qi an d 4>02 are (—2.2487,-1.8116) 
and (1.8120,2.2439) (these were obtained from 100 iterations of the boot- 
strap procedure with m n = 135), and approximate 95% confidence intervals 
for ao, 0o, <7o an d Ho, with standard errors computed using I~ 1 (tml)) are 
(1.6847,1.9823), (-0.1403,1), (0.4093,0.5025) and (15.9102,16.0958). 

In contrast, when we fit a causal AR(2) model to {Xt} by maximizing C 
with s = fixed, we obtain r) = (§i, 9 2 , a, (3, a, /})' = (0.4326, 0.2122, 1.7214, 
0.5849,0.1559,5.6768)'. The sample autocorrelation functions for the ab- 
solute values and squares of the mean-corrected residuals from this fitted 
causal model are given in Figure 5. Because both the absolute values and 
squares have large lag-one correlations, the residuals do not appear indepen- 
dent, and so the causal AR model is not suitable for {A^}. 

APPENDIX 

In this final section, we give proofs of the lemmas used to establish the 
results of Section 3. 

Lemma A.l. For any fixed u € M. p and for Z k j(u) between Z k j and 
Z k>j + [c{ao)) l l aa ooc 3 (u)5 k Tl l/ao , 



(A.i) £EM u )|i\ 

is finite a.s. 



l/a 



dlnf(Z kJ (u);To) dlnf(Z kd ;T ) 



dz dz 
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(a) Z, 




(b) ACF of Absolute Values 
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(c) ACF of Squares 
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(d) Stable QQ-Plot 
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Fig. 4. (a) The residuals {Zt}, (b) the sample autocorrelation function for the absolute 
values of mean- corrected {Zt}, (c) the sample autocorrelation function for the squares of 
mean-corrected {Zt} and (d) the stable qq-plot for {Zt}. 



ACF of Absolute Values 



T" | J J " _ J _ J J _' J _M _" J _ s _ _ _ _' J _ _ _ _ _' J 



10 



20 30 
Lag 



40 



ACF of Squares 




Fig. 5. The sample autocorrelation functions for the absolute values and squares of the 
mean- corrected residuals from the fitted causal AR{2) model. 
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PROOF. Since equation (A.l) equals Y%Li Sj^o l c j( u )l r fc °l^fej'( u ) ~~ 
Z k) j\\d 2 hi f(Z^j(u);T )/dz 2 \, where Z^(u) is between Z k j and Z k j(u), 
(A.l) is bounded above by 



(A.2) 



c(a )] 1/ao a sup 



3 2 ln f(z;r ) 



Oz 



Er fc " 2/ao E 

k=l j^O 



By (2.5) and the continuity of d 2 ln/(-; T )/dz 2 on R, sup 2gR \d 2 In f(z; r )/ 
<9z 2 | < oo. Now suppose k^ £ {2,3, . . .} such that k^ > 2/ao- It follows that 

E{££=fct r fe - 2/Q0 } = ££°=fct r(fc-2/o )A , W < (constat) Er= fc t *r 2/a ° < oo. 
Consequently, since < [^(ao)] 1 ^ 00 ^ < oo, J2j^o c ji u ) < oo by (3.3) and 
EfcLl 1 2/ao < oo a.s., (A.2) is finite a.s. □ 



Lemma A.2. For any fixed u G W, 

i/ao ,-i/ao\ ,91n /(^,j;To) 



(a.s) EE c i( u )( r fe -^" 1/Q0 ) 



(92 



< oo 



a.s. 



Proof. The left-hand side of (A. 3) is bounded above by sup 2eK |51n/(z; 

ro)/dz\ET=i\^ 1/ao -k~ l ^\E^o\cj(u)\. By (2.6), sup 2eK \dlnf(z; t„)/ 
c?z| < oo, by (3.3), J2j^o l c j( u )l < °°; an d, from the proof of Proposition A. 3 

in Davis, Knight and Liu [12], J2T=i l r fc 1/Q ° - k~ l / a °\ < oo a.s. Thus, (A. 3) 
holds. □ 

Lemma A. 3. For any fixed u£W, \ Y,T=iT,j^ c M) 5 kk' l/a °[d\n f{Z kd ] 
To)/dz]\ < oo a.s. 

Proof. The sequence 
series of independent random variables which, by dominated convergence, 
all have mean zero, since J2j^o\ c j( u )\ < °°> su PzeR \9ha.f(z;To)/dz\ < oo 
and E{dln f(Z k j; tq)/ dz} = f^ 00 (df(z;ro)/dz)dz = 0. Therefore, because 



v^r fv^ ( , -i/a <91n/(Z fc i ;T ) 



fe=l 



< sup 



< oo, 



9 In /(z; T ) 



9z 

/ \ 2 oo 

V?yo / fc=i 



the result holds by the Kolmogorov convergence theorem (see, e.g., Resnick [33], 
page 212). □ 
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Lemma A. 4. For u e W and v e M 4 ; 
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El"/(2 t («o + ^, S „);ro + ^) 



(A.4) 



t=p+l 



£ ln/U + rT 1 /^ £ c .( u )Z w ;ro + ^ , 



t=P+i 



j=-oo 

mi/i Z t (-, •) as defined in (2.11), converges in probability to zero on C(IR P+4 ) 
as n — > oo . 



Proof. Let T > 0. We begin by showing that (A.4) is o p (l) on C([-T, 
T]P +i ). Since {Z t (0 ,s )} = {Z t }, and following (3.2), equation (A.4) equals 



(A.5) 



E 

t=p+i 



ain/(Z t * n (u);T + v/v^) 
<9z 



u 



Z *( 00 + "TT7^"' s o ) - z t (e Q ,s Q ) 



n 



u' aZt(0 O) so) 



n 



l/OD 90 



where Z£„(u) lies between Z t (0 o + ra~ 1/a °u, s ) and Z t + n~ 1 / Qo u'aZ t (0 o , so)/ 
90. Equation (A.5) can be expressed as 

ain/(Z* n (u);r + v/^) ^(fl^u), s ) 



1 

2n 2 A*o 



E 

t=p+i 



z 



-u 



90 90' 



-u. 



with #j n (u) between #o an d + 

n -i/«*o u . Following (3.1), the mixed partial 
derivatives of Zt(6,s) are given by 

0, j,k = l,...,r, 

X t+r __j_ k , j = l,...,r, k = r + l,...,p, 
j,k = r+l,...,p, 



d 2 Z t (6,. 



d9jd9k j q 
and so we have 



1 



sup 

( u ',v')'e[-T,T]i>+4 2n z ' a ° 



E 

t=p+i 



<91n/(Z* n (u);T +v/v^) 
<9z 



,d 2 Z t (0 t yu), So ) 

X U — ^— — ; U 



< sup 

2GM,ve[-T,T] 4 



x sup 



<9hi/(z;T + v/v 7 ^) 



9.2 



uef-T^^n^o t E a 



U 50^ U 
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(A.6) 



< sup 



< sup 

zeR,v£[-T,T] 4 



dlnf(z;r + v/y/n) 



Oz 



dlnf(z;r + v/y/n) 



2J2 n p 



n 



2/a 



E El* 

t=p+lj=2 



t-j\ 



Oz 



T 2 p 2 



n 



2/a 



n p oo 

x E E E \4>kZt-j-k\ 



t=p+l j=2 k=—oo 

(recall that X t = Ejl-oo ^j z t-j)- By (2.6), sup zgR ve[ _ TiT] 4 |91n/(z; r + 
v / ' yfn) I '0z\ = 0(1) as n-> oo. Now let e > and k\ = (3/4)ao^{«o < 1} + 
/{ao > 1}) an d observe that E|Zi| Kl < oo and < K\ < 1. Using the Markov 
inequality, 



1 



n p oo 



n 2/a 



E E E l« 



t=p+l j=2 fc=— oo 



>e n 



< 



< 



1 



era 2 /" 
1 



n p oo 



E E E E \^zt-j-k\ 

U=p+1 j=2k=-oo 



n p oo 



E E E E 

U=p+1 j'=2fc=-oo 



era 2 /" 

<e~ Kl n 1 ~ 2Kl/Qo pE|Zi| Kl ^ |^ 



fc=— oo 



0. 



Consequently, (A.6) is 0p (l) on K, and so (A.4) is 0p (l) on C([-T, T] p+4 ). 
Since T > was arbitrarily chosen, for any compact set K C M p+4 , (A.4) is 
Op(l) on 0(iT), and it therefore follows that (A.4) is o p (l) on C(M P+4 ). □ 

Lemma A. 5. For u £ M p and v £ IR 4 , 

n / oo \ 

Y, In/U + n-V* 2 c i (u)^_ i ;r + ^ 

i=p+l \ j=-oo V 71 / 



(A.7) 



£ ln/U + n- 1 /" £ Ci (u)^_ i; ro 



t=p+l 
j n 



3=-oo 



£ — a7 + 2 vI(ro)v 



t=p+i 



converges in probability to zero on C(M. P+4 ) as n- 



oo. 
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Proof. Using a Taylor series expansion about tq, equation (A. 7) equals 



— v 



(A.8) 



t= P +i 



din f(Z t + n-V«o E - _ qo Ci (u)Z w ;T ) 



3r 



91n/(Z t ;r ) 



(A.9) 



9r 

, V " ^ InfjZt + n-V°° E^-oo c 3 - (u)Z f _ j; r* (v)) 
H > „ . v 

t=p+l 
+ ^v'l(r )v, 



<9r <9t' 



where r* (v) is between ro and ro + v/\/ri. Let T > 0. We will show that 
sup u6[ _ T)T]P of 



(A.10) 



— y 



t=p+l 



da 



d\nf(Z t ;r ) 
da 



is o p (l). It can be shown similarly that sup( u / v /y e r t,t]p+ 4 °f (A.8) is o p (l), 
and, using the ergodic theorem, sup/ u / jV /y e r TjT i P +4 of (A.9) is o p (l). Since 
T > was arbitrarily chosen, it follows that (A. 7) is o p (l) on C(IR P+4 ). 
Observe that sup u€ [_ TT ] P of (A. 10) equals 



(A.11) sup 

ue[-T,T]P 



n 



1 A 9 2 ln/(Z-(u);ro) » 



t=p+l 



j=-oo 



where Z£*(u) is between Z t and Z t + n" 1 /^ £°o ^ Cj (u)Z t _j. Follow- 
ing (3.2), there must exist constants C\ > and < -Di < 1 such that 



sup Ic.Hl^d^' 1 VjG {...,-1,0,1,...}, 
ue[-T,T\p 



(A.12) 

and so (A.ll) is bounded above by 
(A. 13) sup 



d 2 In f(z;r ) 


Ci 


dz da 


n l/2+l/a 



n oo 



E E ^'l^-il- 

i=p+l J=— oo 

By (2.7), sup^ gR \d 2 lnf(z;To)/(dzda)\ < oo. Now let e > and «2 = «o(l + 
a /3)/(l + ao/2)/{a < l} + ^{a > 1}, so that K 2 (l/2 + l/a ) > l,E|Zi| K2 < 
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oo and < K2 < 1. Since 



1 



n 00 



1/2+1/ao 



E E * 



i-jl 



f=p+i j=— 00 

00 

< e -«2 n l-K 2 (l/2+lA*0) E | Zl |K2 ^ 



j=-oo 



and „i-Ka(i/2+i/a ) _> , (A. 13) is o p (l) and therefore sup ue[ _ TiT]P of (A.10) 
must also be o p (l). □ 

Lemma A. 6. For u e R p , 



]T ln/fZt + n- 1 /^ £ Cj (u)Z t _ r ,T 



t=p+i 



0=-oo 



(A.14) 



E ln / \ Z t + 

t= P +i \ 



n 



-l/ao 



E 

t= P +i 



co(u) 



In f [ Z t + ^ Z t ; t )- In f(Z t ; to) 



converges in probability to zero on C(MP) as n — > 00. 
PROOF. Equation (A.14) equals 



(A.15) 



co(u) 



n 



l/ao 



E * 



*=p+l 



91n/(Z* (u);r ) 51n/(Z t ** (u); r ) 



where Z t * n (u) is between Z t + ?i~ 1/a ° J2T=-oc c j( u ) z t-j and Z t + ?i~ 1/a ° x 
X;jyo c j( u )^t-j! and ^t*n( u ) is between Z t and (1 + n- 1 / Q °c (u))Z t . For 
T > 0, sup ue [_ TjT ] P of the absolute value of (A.15) is bounded above by 



(A. 16) sup 

ue[-T,T]P 



co(n) A _^ . 2 ln/(Z-(u);ro) 

-27^ E ^E c » z *-i- 



t=p+i jyo 



(A.17) 



+ sup 



c§(u) 



E 



ue[-T,T] P n 2 / Q o 



<9z 2 



2 9 2 ln/(Z;;*(u);r ) 



dz 2 



where Z&» = Z t + n~ l l^ xl >n (u)co(u)Z t + n-V°o A£ n (u) £ i#0 c,(u)Z t _,- 

for some Af n (u), Ajf n (u) 6 [0, 1]. To complete the proof, we show that (A. 16) 
and (A.17) are o p (l). 
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Following (A. 12), equation (A. 16) is bounded above by sup 2gR \d 2 ln/(z;To)/ 

az 2 |n- 2/Q °C?Et%+i I^IE^o^'l^-jl-If^ := (3/4)a /{ao < 1} + /{«o > 
1}, then, for any e > 0, 

P 



> e 



1 

< e -*3 n l-2*s/«0 (E\Zi\ K3 fJ2 (Di 3 ) 1 j| 



which is o(l), and thus (A. 16) is o p (l). 
Equation (A. 17) is bounded above by 



(A.18) sup £ Z, 



+ sup 



cg(u) 



ue[-T,T]P n 



2/a 



(A.19) 



= E Z \ 
t= P +i 



dz 2 



a 2 ln/(Z;-(u);r ) 



dz 2 

d 2 ln/([l + Al, n (u)c (u)/n 1 /"o ]Zt . Tq ) 



dz 2 

and (A.18) is bounded above by sup 2gK |z 2 [<9 2 ln/(z; tq)/8z 2 ]\ x 

sup uG[ _ TT ] P |c (u)| < oo and, from (2.5), sup 2eM \z 2 [d 2 lnf(z;T )/dz 2 ]\ < oo, 
(A.18) is o p (l). An upper bound for (A.19) is 



sup 



c§(u) 



E 4 



ue[-T,T]p n 3 /«o t=p+1 



d 3 ln/(Z t , n (u);ro) 



E c i( u ) Z <-J 



< sup 



3 3 ln/(z;r ) 



dz 3 



n \ 3 n 

E ^ 2 EM Jl i^i, 

t=p+l 3^0 



n l/a 



where Z^ n (u) is between Z t ***(u) and [1 + \\ n (u)co(u)/n l / a °]Z t . If K4 := 
3ao/8, then, for any e > 0, 

n 1 K 4 \ 



1 



3/o 



n 



> e 



K.-4 



t= P +i 3^0 

< e-' t4 n 1 - 3K4 / Q0 E{Z 1 2 ' t4 }E|Zi|' t4 ^(D* 4 )^ ™ 0. 

j¥0 
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Since sup zeK |9 3 In/ (2; To)/dz 3 \ < 00 (see DuMouchel [17]), it follows that 
(A.19) is also o p (l). □ 

Lemma A. 7. For u = (u\, . . . , u p )' £ W, 



(A.20) 



E 

t= P +i 



+ (n — p) In 



-l/ao 



U, 



?0p 



I{s > 0} 



converges in probability to zero on C(MP) as n— > 00. 

PROOF. If so = 0, the result is trivial since, from (3.2), cq(u) = u p 6q^I{sq > 
0}, and so, when sq = 0, equation (A.20) equals zero for all u E MP. Now con- 
sider the case so > 0. Choose arbitrary T > and note that sup ue [_ TT ]p °f 
the absolute value of (A.20) equals 



sup 

ue[-T,T]P 



E 

t= P +i 



(A.21) 



cq(u) dln/(Z f ;T ) 

cg(u) .5 2 ln/(4 >n (u);r ) 



+ 



2n 2/a Z * 



+ (n — p) In 



7Qp 



where Zj n (u) is between Z t and [1 + n 1//Q °co(u)]Z t . Equation (A.21) is 
bounded above by 



(A.22) 
(A.23) 
(A.24) 



sup 

ue[-T,T]P 



c d( u ) 



l/"0 



t=p+l 



1 + 3 



91n/(Z t ;r c 



+ sup 

uG [-T,T]i 



+ sup 

ue[-T,T]P 



(n — p) 
c§(u) 



co(u) 



)1 



l/a 



111 



7Qp 



■2 

2n 2/a ^ " 
t=p+l 



E 3 



5 2 ln/(4 n (u);ro) 



we complete the proof by showing that each of these three terms is o p (l). 

Since {1 + Zt[d\nf(Zt; tq)/8z\\ is an i.i.d. sequence with mean zero 
(which can be shown using integration by parts) and finite variance, 

dln/(Z t ;r r 



E< 



n 



1 

l/ao 



E 

t=p+i 



i + z t - 



dz 
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^ „L „dlnf(Z t] T )" 



2/a 



n 



£ E 1+^ 



t=p+i 



<9z 



which is o(l). Therefore, because sup ue [_ rr ] P |co(u)| < 00, (A. 22) is o p (l). 
Next, (A.23) equals sup ue[ _ TjT]P \(n-p)[rr 1 ^ a °u p e^-hi\l+n- 1 / ao u p e i 



which is o(l). And finally, (A. 24) is bounded above by 



P w 0p 



sup 

zeR 



, d 2 Inf(z-To) 



< sup 

zGR 



9z 2 
2 <9 2 ln/(z;r 



ue[ SU T P r]P 2n2/«o 



2 



dz 2 



n i-a/oo sup 

ue[-T,T]P 



4n(u) 

c§(u) 



^Op 



1/00 



which, since sup zGR |z [9 ln/(z; To)/dz ]| <oo, is also o(l). □ 

Lemma A. 8. For any fixed u£MP andveM 4 , (Wt(u),T n (v))' 4. (W(u). 
v'N)' on M 2 as n— > 00, wwtft W(u) and v'N independent. [W^(-), T n (-) and 
W(-) were defined in equations (3.8), (3.9) and (3.4), respectively, and, from 
Theorem 3.3, N ~ N(0, I(t )).J 

Before proving this result, we introduce some notation and three addi- 
tional lemmas which will be used in the proof. First, define a set function 
e x (-) as follows: e x (A) = I{x E A}, and, for m > 1, let 

ex^fV^l, 0,...,0 ),..., e m = ( 0,...,0, 1) 

m times m— 1 times 2m— 1 times 

and 

e-i^t^O, l,0,...,0),...,e_ m = (l, 0,...,0 ). 

m— 1 times m times 2m— 1 times 

Now define 

n 

S m , n (-) = 2J e (Z t ,[c( Qo )]' 1/a o<7 - 1 n- 1 /«o(Z t+m ,...,Z t+1 ,Z t _ 1 ,...,Z t _ m ))(") 

and 



t=P+i 



S -»(')-EE( £ (Z t ,_ ) ,e_ j 8 t r fc 1/ "»)(') + ^. .-.A.r-V<*^(-))- 



fc=lj=l 

By the following lemma, S m)n {-) can converge in distribution to 5 m (-). 



2 m 

Lemma A. 9. For any fixed relatively compact subset A of R x (R \ 
{0}) fa subset A for which the closure A is compact; note that a compact 
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subset ofW \{0} = [ — oo, oo] 2m \ {0} is closed and bounded away from the 
origin) of the form 

A = {a ,b ] x (a_ m , 6_ m ] x • • • x (a_i, 6_i] 

(A.25) 

x (ai,h] x ••• x (a m ,b m ], a^bj V|i| G {l,...,m} 

and /or any /ixed v G M 4 , (5 m ,„(A), T n (v))' (5 m (i4), v'N)' on R 2 asn^ 
oo, with S m {A) and v'N independent. 

Proof. Let Ai,A2 G R. Following Theorem 3 on page 37 of Rosen- 
blatt [34], this Lemma holds if cumfc(AiS , TTljn (A) + A2T n (v)) — ► cumfc(AiS m (yl) + 
A2v'N) for all k > 1, where cumfc(AT) is the /cth-order joint cumulant of the 
random variable X. So, 

cumfc(X) = cum(X, . . . , X). 

k times 

Note that since S m (A) and v'N are independent, cum.k(XiS m (A) + A2v'N) = 
Af cum fc (S m (i)) + A|cum fe (v'N). 

Fix k > 1 and denote the A;th-order joint cumulant of i Xs and j Fs 
(i + j = k) as cumjj (X, F) . So, 

crnrn^X, F) = cum( X,...,X , F, . . . , F). 

i times j times 

Then, by linearity, 

cum k (XiS m!n (A) + A 2 T n (v)) = Af cum k (S m , n (A)) + Ag cum fc (T n (v)) 

+ E ( 7 ) AiAa - - 7 cum Jjfc _ j (S' min (^),r n (v)). 
j=i V J ' 

Also by linearity, for j G {1, . . . , k — 1}, 
cum iifc _ j (S m n (A) , T n (v) ) 

(A.26) 

n n 

= E "' 51 cum(y ilin ,...,y tjin ,w tj+lin ,...,M / tfein ), 

*l=p+l tfc=p+l 

where V t , n := £ (Zti[£(Qor i/.o (J -i n -i/ a o(z t+m ,...,z t+1 ,z t _ 1 ,...,z t _ m ))( j4 ) and ^,« : = 
n~ 1 ' 2 \ J dlnf(Zt] to) /dr. Due to the limited dependence between the vari- 
ables {Vt, n }™=p+i> {Wt.nltLp+n equation (A.26) equals 



n 



E E - E E 

tl=p+l |t 2 -ti|<2jm |i i -ti|<2j»n |t j+ i-ti|<(2j+l)m 

(A.27) 

^ cum(V tlin ,...,^ in ,M / tj+lin ,...,W tfe ,n); 
|t fc -ti|<(2j+l)m 
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this sum is made up of (n —p)(4jm + 1 ([4j + 2]m + l) k ~i terms. There- 
fore, since \Vt, n \ < 1, and E|Wt, n | < oo for all t > 1 and all n, (A. 27) is o(l) 
if k — j > 3 [as a result of the scaling by n~ J')/ 2 ], Equation (A. 27) is also 
o(l) for Ar-j G {1,2} if nE|14 1:n W t2 , n | = o(l) and nE\V tl , n Wt 2 , n Wt 3 ,n\ = o(l) 
for any ti,t 2 ,t^- We will show the limit is zero in one case; convergence to 
zero can be established similarly in all other cases. 

2 m 

Since A is a relatively compact subset of R x (R \ {0}), at least one of 
the intervals (a_ m , 6_ m ], . . . , (a_i, (ai, &i], . . . , (a m , 6 m ] does not contain 
zero. We assume (o_i, 6_i] does not contain zero and show that nE|Vi in W2,n| - 
o(l). First, from (2.5)-(2.10), there exist constants C V ,D V < oo such that 
|v'<91n/0;T )/dT| < C v + D v \z\ a °/ 4 Vz G R. Hence, because 

= e (Z 1 ,[c(a )]- 1 /-o ( 7 - 1 n- 1 /-o(Z 1+m ,...,Z 2 ,Zo,...,Z 1 _ m ))(^) and = 

n- 1 /2 v /ain/(Z2;T )/ar, 
nE\V 1:n W 2:1l \ 



< rc x / 2 E 



< an 1/2 P(|Z 2 | > n^C) + D v n 1 / 2 E{\Z 2 \ ao/i I{\Z 2 \ > n^C}}, 

where ( := [c(a )] 1/Q °o-omin{|a_i|, |6_i|}. By (2.3), since C > 0, n 1 / 2 P(|Z 2 | > 
ra V a o£) _> o, and, using Karamata's theorem (see, e.g., Feller [19], page 283), 
n 1 / 2 E{\Z 2 \ ao / 4 I{\Z 2 \ yn 1 / ^}} < (constant)^/ 2 '(n 1 ' /ao e) ao/4 P(|Z 2 | ^n^C), 
which is o(l) by (2.3). 

It has therefore been established that cunifc(Ai S m>n (A) + \ 2 T n (v)) = x 
cum.k(S m!n (A)) + A| cumfc(T n (v)) + o(l) for arbitrary k > 1. Following the 
Proof of Lemma 16 in Calder [7], it can be shown that cum/ c (S' m)n (^4)) — > 

cum.i t (S m (A)). Note that, from Davis and Resnick [13], S mjn (A) — > S m (A) on 
R and S m (A) is a Poisson random variable, so all cumulants are finite. It is 
relatively straightforward to show that cunifc(T n (v)) — ► cuHifc(v'N) (see the 

Proof of Lemma 16 in [7] for details), which is not surprising since T n (y) — > 
v'N on R by the central limit theorem. Consequently, cumfc(Ai5 mjn (A) + 
X 2 T n (v)) — > \\ cumfc(5 m (^4)) + X 2 cum^v'N), and the proof is complete. □ 

Lemma A. 10. Let U t ~ n (u) = n" 1 /" c -j( u ) z t+j, U^ n (u) = n - l l a ° x 
E^c^u)^- and I^< M = I{\Z t \ < M}I{(\U t - n (u)\ > A) U (|*7+»| > 
A)}. For any fixed u G MP and any k > 0, lim A ^ + h m M^oohmsup n ^ 00 of 

n 

£ {[lnf(Z t + U t - n (n) + U+ n (u);r ) 
t= P +i 

(A.28) 

-lnf(Z t ;r )][l-lt X ' M }} : 
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is zero. 



PROOF. Note that, for any t £ {p + l, . . . ,n} and any n, ]nf(Z t + U t n (u) + 
C/+(u);T )-ln/(^;T ) = [^ 

U£ n (u)} 2 [d 2 In / (Z t * n ; T )/dz 2 ]/2, where Z t * n lies between Z t and Z 4 + U^ n {u) + 
f7 t + n (u). Note also that 

1 - I^ M = I{\U t - n (u)\ < \}I{\U+ n (u)\ < A} + J{|Z t | > M}I{\U t - n (u)\ > A} 

+ /{|Z t |>M}I{|tf+ n (u)|>A} 

- I{\Z t \ > M}/{|^(u)| > A}/{|[/+ n (u)| > A}. 
Consequently, (A. 28) is bounded above by 



E 

t=p+i 



(«Tn(u) + ^+(u)) 81n/ ^ ;To) /{|^-(u)| < A} 



x/{|C/+(u)|<A} 



> 





a 2 ln/(z;r ) 


sup 

VzeM 


9z 2 



x E [|^(u) + C/ t +(u)| 2 /{|C/^(u)|<A}/{|[/+ n (u)|<A}]> 

t=p+l 



+ P (J {(|z t |>M)n(|[/ t -(u)|>A)} 



U=p+1 

/ n 



+ 2P (J {(|Z i |>M)n(|C/+,(u)|>A)} . 
\t=p+i / 

The proof of Proposition A.2(a)-(c) in Davis, Knight and Liu [12] can be 
used to show that lini; v _ > o+ liniM— >oo hm sup n _ +oc of each of the four sum- 
mands is zero. □ 

Lemma A.ll. Letl^f = I{\Z Kj \ < M}/{|[c(Q )] 1 / Qo aoc J (u)^r- 1/Qo | > 
A} . For any fixed u£K p , 



^^[{ln/(Z fcii + [c(ao)] 1/a Vo Ci (u)5 fe r fe - 1/Qo ;To) 



(A.29) 

converges in probability to zero as A — > + and M — > oo. 



ln/(Z fcj ;r )}(l-4 A i f)] 
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Proof. The absolute value of (A. 29) is bounded above by [c(ao)] 1//Q ° x 
a sup, eR |dln/(z; r )/dz| £r=i ^ 

oo a.s., since E{r~ 1/a °} = 0{k~ 1 t aa ) for k > 1/cto- Thus, the result holds if 
«o < 1- 

For ao > 1, the proof of this lemma is similar to the Proof of Lemma A. 10. 
We omit the details. □ 

We now use Lemmas A.9-A.11 to prove Lemma A. 8. 



Proof of Lemma A. 8. By Lemma A. 9, for any relatively compact sub- 
set A of M x (I 2m \{0}) of the form (A.25) and any v G R 4 , (S m , n (A),T n (y))' - 
(S m (A),v'~N)' on R 2 , with S m (A) and v'N independent. It can be shown 
similarly that, for any £ > 1 and any relatively compact subsets A\, . . . ,A# 
of R x (l 2m \ {0}) of the form (A.25), 

(A.30) (S^CAi), • • • , S m!n {A e ),T n (v))' 4 (5 m (Ax), . . . , S m (At), v'N)' 
on R i+l , with (S m (Ai), . . . , S m {Ai))' and v'N independent. Now, for fixed 



E^ic-^u)^ and [/+(u)= W - 1 / a »Ef=iC J (u)2 H , and let 



ueR p , let S n (-) = E?^ P +ie(z t) D , t - n (u) I D'+ n (u))(0» with u t,n( n ) = n 



- £( e (Z lb ,_ j) [c(aD)] 1 /^ on c_ J -(u)« fc r; 1/a o,0)^ 

fc=lj'=l 



tj ,0,[c(ao)] 1 /°Oa oCj (u)J t r- 1/a «)('^ 



Following the proof of Theorem 2.4 in Davis and Resnick [13], using (A.30), 
the mapping 

j z t+mi ■ • ■ > z t+li Zt—li ■ ■ ■ i Zt-m) 

(m m \ 

t Jc(a o )] 1/Q Vo^c_ j (u)^ +j Jc(«o)] 1/Q Vo^c J -(u)^ , 
3=1 j=l J 

and by letting m — > oo, it can be shown that 

(A.31) (S„(ii), . . . , S n (A e ),T n (-v))' ± . . . , S(^), v'N)' 

on R f+1 , with (,S(^4i), . . . , S{Af))' and v'N independent, for any relatively 
compact subsets A±, . . . ,Ag of R x (R \ {0}). 

Since (5 n (Ii), . . . , 5„(4«))' ^ O^l), • ■ • , S(A e ))' on R< for arbitrary £ > 
1 and arbitrary, relatively compact subsets A\, . . . , Ag of R x (R \ {0}), 

n 

~g(Z t ,U t - n (u),U+ n (u)) 

t=p+l 
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k=ij=i 



+ ~g(Z k j,0,[c( ao )] 1/ao aoc J (u)5 k r i ; 1/ao )) 



on M for any continuous function g on K x (R \ {0}) with compact support 
(see Davis and Resnick [13]). Because it is almost everywhere continuous 

on I x (1 \ {0}) with compact support, we will use g(x,y,z) = [ln/(x + 
y + z;t ) - In f(x;r )]I{\x\ < M}I{(\y\ > A) U (\z\ > A)}, where M,A>0. 
By Lemma A. 10, for any k > 0, lim^_ i . + limj^^ 0O limsup n _ >(X3 P(|W^(u) — 
Et= P+ i 9(Zt, U t ~ n (u), U+ n (u))\ >k)=0 and, by Lemma A.ll, 

oo oo 

EE(»(Vi. [c"(oo)] 1/oi «^.i(u)«»r fc " 1/ao ,o) 



k=tj=i 



+ g(Z kJ A[c(ao)] 1/ao ^c J (n)5 k T- 1/ao ))^W(u) 



as A — > + and M — > oo [W^(-) and W(-) were defined in equations (3.8) 
and (3.4), resp.]. Therefore, by Theorem 3.2 in Billingsley [2], it follows 

from (A.32) that W^(u) — > W(u) on M for fixed u € M p , and consequently 
the result of this lemma follows from (A. 31). □ 

Lemma A. 12. For any T > and any k> 0, 



(A. 33) lim limsupP 

e—>0+ n— »oo 



sup \Wi(u)-Wl(v)\> K )=0. 

lull,l|v||<T.||u-v||<6 



[W^(-) was defined in equation (3.8).] 

PROOF. For u, v G M p , 
|Wt(u)-Wt(v)| 



n 



^ E fe^(u-v)^ 



d\nf(Zl n (u,v)-r c 



t=p+l \jji0 



< 



1 



71 



1/ao 



+ ( sup 

1 



E (e c ;( u - v )^- 

t=p+l \j^0 
d 2 \nf(z;r ) 



dz 

d\nf(Z t -To) 



dz 



dz 2 



n 2/a 



E 

t= P +i 



E c i( u " v ) Z *-i 



E c i( u ) z *-j 



+ 



E c j( v ) z *-j 
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where Z t *„(u,v) lies between Z t + n~ 1 / a °'£j 1 to c j( u ) Z t-j and Zf + n' 1 / 00 x 
J2j- / L Cj(v)Zt-j . Following the Proof of Theorem 2.1 in Davis, Knight and 
Liu [12] (see page 154), if {7Tj}j^o is a geometrically decaying sequence, 
then it can be shown that n~ l / a ° J2?= P +i(J2j^o^j z t-j)[dlii f(Z t ;To)/dz] = 

O p (l) and n- 2 /°° EtLjH-iCEjyo \^i z t-j\f = O p (l). Therefore, by (A.12) and 
because Cj(u) is linear in u for all j, (A. 33) holds. □ 

Lemma A. 13. //, as n — > oo, m n — > oo with m n /n — > 0, then for any 
T > and any k > 0, 



(A.34) 



P sup \W mn (u)-WUu)\>K\X 1 ,...,X n )^0 



u <T 



[Wh (•) and W„ ln (-) were defined in equations (3.14) and (3.15).] 



con- 



Proof. Choose arbitrary T, k > 0, and let the sequence {ipj}j?__ 
tain the coefficients in the Laurent series expansion of l/[9y [ - L (z)9l Ah (z)]. 
From (3.11), for t G {l,...,m„}, ^(5)^(5)^* = 2?, and so X t * = 
J^j^-ooi'j^t-j- From Brockwell and Davis [6] (see Chapter 3), there ex- 
ist C2 > 0, < Z?2 < 1 and a sufficiently small <5 > such that, whenever 

II^ml — #o|| < 8, \tpj\ < C2-D2 ' and also sup|| u || <T |cj(u)| < C2-D2 f° r au 
jf € {. . . , —1,0, 1, . . .} [the Cj(u)s were defined in (3.13)]. Now observe that 
the left-hand side of (A.34) is bounded above by 



(A.35) 



Pf sup \W mn (u)-Wi n (u)\> K \X 1 ,...,X n )l{\\6 ML -e \\<5} 

\\\u\\<T / 



+ i{\\o ML -e \\>5}, 



and that /{||#ml — #o|| > <5} is o p (l) since 0ml - > #o- For u = (ui, . . . ,u p )' G 



W" mre (u)-M (u) 



E 

f =;-'+! 



in/ z; ML + 



u 



m 



l/a 



so ;tml - ln/(Z t *;TML) 



E 

t=p+i 



In/ Z t * + m" 1 / 00 ]T c»Z t *_,,; r - ln/(Z t *;T„) 



+ (m n - p) In 



O . — 1/cJO 

7p,ML + m„ Up 



7p,ML 



/{so>0}, 
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and so, using arguments similar to those given in the proofs of Lemmas A.4- 
A.7, it can be shown that the first summand of (A. 35) is also o p (l) if, for 
any e > 0, 



(A.36) 
(A.37) 

(A.38) 

(A.39) 
and 

(A.40) P| 



Co 



m n oo 



2 /ao E E D^\Z^\>e\ Xl ,...,X n \, 
,rn n t=p+l j=—ao / 



sup \T iM L-Toi 
ie{i,...,4} 

,b1i 



Co 



in 



l/ao 



x E E D^\Zl j \>e\X 1 ,...,X n \, 

t=p+l j=— oo / 



Co 



£ \Z;\J2D l o j] \Z:_ j \>e\X 1 ,...,X n 



2/a 

m n t=p+i j^o 
C 2 



s /ao E (Zt?Y,Df\Zt. 3 \>e\X^...,X n 
,m n t=p+i jjto j 



E 



, m n t=p+l 



l + Zl 



ain/(Z t *;T ) 



Oz 



> c\Xi, . . . ,x n 



are all o p (l). To complete the proof, we show that (A.36) and (A.40) are 
both o p (l). Since u^^tml — To) = O p (l) and m n /n — > 0, using the Proof 
of Lemma A. 5, it can be shown similarly that (A.37) is o p (l). The Proof of 
Lemma A. 6 can be used to show that (A.38) and (A.39) are o p (l). 

Recall, from the Proof of Lemma A. 4, that k\ = (3/4)ao/{ao < 1} + 
I{ao > 1}. By the Markov inequality, equation (A.36) is bounded above 
by 



E W) w 



J=-oo 



E{|z;nx l5 ...,X re }; 



this is o p (l) since mk 2ki//q ° — > and, using 6ml —> #o and E|Zi| Kl < oo, it 
can be shown that E{\Zf\«i\Xi,...,X n } = (n - p)~ l Et= P+ i \Z t {0 M L, s )\ Kl 
is O p (l). 

We now consider (A.40), which is bounded above by 

7 ,d\nf{Z* t ;T Q y 2 



(A.41) e -Xr 2/o!o Ej (l + Z x 

(A.42) 



Oz 



X 



+ € 



m" - m r 



-1 "'n 



2/a 

mn 



1 + 2 ;»!)| Xl x, 
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Since m n 



l-2/ao 



and, by (2.6), sup zgR \z[dlnf(z; T )/dz]\ <oo, (A. 41) is 



o p (l). Now consider 

(A.43) E {i + z;- ain/(z * ;ro) 



Oz 



I X]_ , . . . , X n 



(A.44) 



(A.45) 



1 



n — p 



t=p+i 



E i+s 



din f(Z t -T 

dz 



+ 



1 



n — p 



ST v la N91n/(Z i (0 M L,So);T O ) 
2^ ^(6>ML,S ) TT 



t=p+l 



E * 

t=p+i 



C?2 

gln/(Z f ;r ) 



By the central limit theorem, (A.44) is O p (n x / 2 ). In addition, since Z t 
Z t (6 ,s ), (A.45) equals 



(A.46) 



(0 ML - #0/ ^ 

re — p . i 



din f(Z t (0* n ,s );To) 



Oz 



+ Z t (9* n ,s 



d 2 lnf(Z t (O* n ,s )-T ) 

dz 2 



dz t (e* n , So ) 

dO 



with 0* n between Oml and Oq, and, because sup zeR | [din f(z; To)/dz] + z[d 2 x 
ln/(z; tq)/8z 2 ]\ < oo, the absolute value of (A.46) is bounded above by 



(A. 47) (constant) sup 



7j,ML 



E 



;e{i,..., P }V n ~P t= P +i 



dZ t (0* n ,s ) 



Recall, from the Proof of Lemma A. 5, that K2 = ao(l + ao/3)/(l + ao/2)I{ao < 
1} + I{a > 1}. For i G {1, . . . ,p} and e > 0, 



(n _ ^1/2+1/00 i z. i 



dZ t {0* n ,s ) 



dO, 



/{||0ML-0 O ||<5} 



K 2 



> e 



K 2 



< e _K2 (n — p) 1 ~ fi2 ( 1 / 2+1 / ct °)E 



dZ t (0* n ,s ) 



ddi 



ni|0ML-6>o||<<5}k 



which can be shown to be o(l) for sufficiently small 5 > since At2(l/2 + 
l/a ) > 1 and E|Zi| K2 < oo. Therefore, since n 1 /" (0 M L - 0o) = O p (l), it fol- 
lows that (A.47), and hence (A.45) and (A.46), are o p (n~ 1/2 ), and so (A.43) 
is O p (n~ 1 / 2 ). Since m n /n — ► 0, (A. 42) must be o p (l), and so the proof is 
complete. □ 
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